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Abstract 

We  examined  individual  differences  in  interacting  and  learning  from  diagrams, 
multimedia  presentations  and  hypermedia  instructional  manuals  and  how  diese  individual 
differences  related  to  spatial  abilities  and  knowledge.  In  several  experiments,  we  found 
that  comprehension  of  mechanical  systems  from  static  and  animated  diagrams  was  quite 
limited,  and  often  resulted  in  misconceptions  about  how  the  systems  worked.  Students 
learned  more  from  static  and  animated  diagrams  if  they  were  augmented  with  verbal 
instruction,  and  students  with  high  spatial  ability  and  more  background  knowledge 
learned  more  from  these  multimedia  presentations  than  students  with  low  ability  and  less 
knowledge.  Li  all  of  our  experimeiits,  learning  from  static  diagrams  and  text  was 
equivalent  to  learning  from  animated  diagrams  and  commentaries,  when  these  materials 
contained  the  same  information.  There  was  no  evidence  that  different  formats  of 
iiistruction  were  more  or  less  effective  for  individuals  with  different  abilities  and 
knowledge. 
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2.  Co-Principal  Investigator 
none 

3.  Long  Term  Goals  of  the  Project 

3.1  To  examine  how  (1)  spatial  ability  and  (2)  knowledge  affect  learning  from  diagrams, 
multimedia  and  hypermedia  displays. 

3.2  To  examine  individual  differences  in  how  people  interact  with  hypermedia  displays. 

3.2  To  translate  this  knowledge  into  practical  guidelines  for  designing  instructional 
materials  for  people  of  different  abilities  and  expertise. 

4.  Scientific  and  Technical  Objectives 


4. 1  To  understand  how  people  with  different  levels  of  spatial  ability  and  expertise 
comprehend  diagrams  and  multimedia  displays  in  domains  such  as  mechanics  and  what 
they  learn  from  these  displays/  ^  ^  ^  ^  ^ 

4.2  To  develop  design  principles  for  interactive  displays  in  these  complex  domains  and  to 
build  and  evaluate  prototypes  embodying  these  principles. 

5.  Approach 

5. 1  Designing  different  versions  of  interactive  and  displays  that  explain  how  complex 
mechanical  systems  work. 

5.2  Conducting  empirical  studies  of  how  people  interact  with  these  displays,  what  they 
learn  from  them  and  from  parts  of  the  displays  (e.g.,  static  vs.  animated  diagrams), 
and  the  extent  to  which  they  can  apply  wW  they  learned  to  make  inferences  and 
solve  problems. 

5.3  Examining  the  effects  of  individual  difference  variables  such  as  spatial  abilities  and 
knowledge  on  how  and  what  people  learn. 

6.  Concise  Progress  Summary 

In  research  conducted  with  funding  from  the  Office  of  Naval  Research  in  the  last 
6  years,  we  developed  a  model  of  how  people  understand  and  make  inferences  from 
multimedia  presentations  (Narayanan  &  Hegarty,  1998;  2002).  The  focus  of  the 
AASERT  grant  was  to  further  test  and  develop  this  model  by  conducting  empirical 
studies  of  how  mdividuals  with  different  amounts  of  spatial  ability  and  knowledge 
comprehend  and  make  inferences  from  these  presentations.  These  studies  mvolved 
collection  ^d  analysis  of  several  different  types  of  data,  including  measures  of  learning 
and  comprehension,  measures  of  how  people  interact  with  multimedia  and  hypermedia 
displays,  response  times,  eye-fixation  protocols,  and  verbal  protocols.  Most  of  our  work 
was  in  the  domain  of  mechanics 

We  conducted  research  on  comprehension  of  three  different  types  of  materials: 

1.  Static  and  animated  diagrams. 

2.  Multimedia  presentations,  including  either  static  diagrams  accompamed  by  text  or 
animated  diagrams  accompanied  by  verbal  commentaries. 

3.  Hypermedia  instructional  manuals  that  included  static  and  animated  diagrams, 
written  text  and  aural  commentaries. 

6.1  Research  on  Comprehension  of  Diagrams 

In  one  series  of  experiments,  we  examined  what  students  learn  from  static  and 
animated  diagrams.  Although  the  focus  of  the  research  project  was  on  comprehension  of 
multimedia  and  hypermedia  displays,  it  was  important  to  focus  on  comprehension  of 
diagrams  alone,  because  a  theory  of  comprehension  of  multimedia  presentations  must 
include  an  account  of  the  cognitive  processes  and  comprehension  outcomes  that  result 
from  exposure  to  the  individual  media  that  make  up  multimedia  presentations.  This 
scientific  account,  in  turn,  can  be  the  basis  of  a  set  of  principles  for  the  design  of 


multimedia  presentations  (cf.  Narayanan  &  Hegarty,  1998;  2002),  for  example  by 
specifying  when  and  how  diagrammatic  materials  should  be  augmaited  by  text. 

We  conducted  4  experiments  on  comprehension  of  diagrams  of  mechanical 
systems.  In  Experiment  1  we  compared  comprehension  of  a  mechanical  system  after 
Students  studied  a  single  static  diagram  of  a  mechahicar  system,  a  series  of  three  diagrams 
that  showed  different  phases  in  the  operation  of  the  mechanical  system,  and  an  animation 
of  the  mechamcal  system.  Those  who  learned  from  3  diagrams  or  an  animation  had  better 
comprehension  of  the  system  than  those  who  studied  a  single  diagram,  however  there  was 
no  significant  difference  in  comprehension  between  the  3-diagram  group  and  the 
animation  group  (Kriz,  2002;  i  Hegarty,  Kriz  &  Gate,  2003).  There  were  no  effects  of 
spatial  ability  on  comprehension. 

Experiment  1  therefore  indicated  limited  effectiveness  of  an  ammation,  in  that 
studying  a  complete  ammation  with  about  140  frames  lead  to  the  same  amount  of 
understanding  as  studying  3  frames  of  the  animation.  The  literature  on  animations  (e.g., 
Tversky,  Morrison  &  Betrancourt,  2002)  suggested  two  possible  reasons  why  our 
animation  might  not  have  been  effective: 

(1)  The  animation  was  not  interactive,  and  it  is  possible  that  students 
comprehension  outcomes  could  not  keep  up  with  the  speed  of  the  animation. 

(2)  The  animation  showed  several  components  moving  at  once  and  there  were  no 
attentional  cues  to  direct  smdents’  attention  to  the  most  important  information. 

We  conducted  2  further  studies  in  which  we  compared  different  types  of 
animation  to  learmng  from  a  single  static  diagram.  In  Experiment  2  we  compared 
comprehension  of  an  mteractive  ammation,  a  non-interactive  animation  and  a  static 
diagram.  Students  who  studied  the  animations  had  better  comprehension  than  those  who 
studied  the  static  diagram,  but  there  was  no  significant  difference  between 
comprehension  of  the  two  animations.  In  Experiment  3  we  compared  an  interactive 
animation  with  attentional  cues,  an  interactive  animation  without  attentional  cues  and  a 
single  static  diagram.  Again  students  who  studied  the  two  animations  had  better 
comprehension  than  students  who  studied  the  static  diagram  but  there  was  again  no 
difference  in  comprehension  between  the  two  animation  groups.  There  were  no  effects  of 
spatial  ability,  or  interactions  of  ability  with  form  of  instruction  in  either  experiment. 

In  Experiment  4,  we  monitored  students’  interactivity  and  eye-fixations  while 
they  studied  interactive  animations  either  with  or  without  attentional  cues.  Examination 
of  the  interactivity  data  indicated  that  students  used  the  interactivity  provided. 
Examination  of  their  eye  fixations  indicated  that  they  were  more  likely  to  look  at 
components  of  the  animations  when  ftiey  were  directed  to  these  components  by 
attentional  cues.  However  there  were  no  effects  of  either  interactivity  or  attentional  cues 
on  their  comprehension.  Experiments  2-4  are  currently  being  written  up  for  publication 
along  with  some  later  research  that  was  funded  by  a  subsequent  grant. 

fri  a  final  experiment  on  comprehension  of  diagrams,  we  obtained  preliminary 
eye-fixation  evidence  for  mental  animation  as  a  strategy  in  mechanical  troubleshooting. 


Previous  studies  of  mechanical  reasoning  suggest  that  people  formulate  a  mental  model 
of  a  mechanical  system  and  are  able  to  manipulate  this  model  while  reasoning  about  the 
system.  Much  as  one  can  rewind  or  advance  a  video,  people  seem  to  be  able  to  “play”  a 
causal  sequence  of  steps  in  their  minds.  This  internal  procedure  has  been  dubbed 
‘mental  animation’  (Hegarty,  1992).  Taking  an  embodied  cognition  approach,  which 
suggests  that  our  bodies  are  involved  in  cognitive  processes,  we  proposed  that  eye 
movements  can  provide  evidence  for  internal  processes  such  as  mental  animation.  This 
study  analyzed  eye  movement  data  from  seven  subjects  who  learned  about  a  flushing 
cistern  from  a  computer  animation.  After  viewing  the  animation,  subjects  were  presented 
with  troubleshooting  questions  that  required  them  to  think  about  the  reasons  why  a 
breakdown  would  have  occurred.  While  answering  these  questions,  subjects  were 
allowed  to  view  a  static  diagram  of  the  cistern  in  resting  position.  The  eye  movement 
data  show  distinct  viewing  patterns  accompanying  certain  responses,  including  fixation 
patterns  that  mirror  the  movements  presented  in  the  computer  animation.  Furthermore, 
subjects  often  directed  their  gaze  to  empty  spaces,  where  parts  would  have  been  had  they 
moved.  While  talking,  subjects  did  not  simply  look  at  the  referent  of  their  response. 
Rather,  their  eye  movements  indicate  that  they  were  imagining  mechanical  movement 
and  not  simply  focusing  visual  attention  on  the  part  being  discussed.  These  results 
suggest  that  subjects  were  using  mental  animation  while  reasoning  about  faults  in  the 
system;  that  while  viewing  a  diagram,  subjects  were  processing  the  static  visual 
information  in  a  dynamic  manner.  This  research  was  presented  at  the  European  Eye 
Movement  Conference  in  2003  (Kriz  &  Hegarty,  2003). 

6.2  Research  on  Comprehension  of  Multimedia  Presentations 

Research  on  comprehension  of  diagrams  alone  indicated  that  diagrams  lead  to 
limited  and  often  erroneous  comprehension  of  how  mechanical  systems  work.  In  another 
series  of  experiments,  we  examined  comprehension  of  multimedia  displays  that  included 
diagrams  and  accompanying  verbal  instruction. 

In  a  series  of  four  experiments  (Hegarty,  Narayanan  &  Freitas,  2002;  Hegarty, 
Kriz  &  Cate,  2003)  we  examined  the  effects  of  learning  from  external  computer 
animations  and  mental  animation  of  static  diagrams  on  people’s  mental  models  of  a 
mechanical  system.  In  three  experiments,  students  learned  how  a  mechanical  system 
works  from  various  instructional  treatments  including  viewing  a  static  diagram  of  the 
machine,  predicting  motion  from  static  diagrams,  viewing  computer  animations,  and 
viewing  static  and  animated  diagrams  accompanied  by  verbal  commentaries.  Although 
students’  understanding  of  the  system  was  improved  by  viewing  both  static  and  animated 
diagrams,  there  Was  no  evidence  that  animated  diagrams  led  to  superior  understanding  of 
dynamic  processes  compared  to  static  diagrams.  Comprehension  of  diagrams  was 
enhanced  by  asking  students  questions  that  required  them  to  predict  the  behavior  of  the 
machine  from  static  diagrams  and  by  providing  theni  with  a  verbal  description  of  the 
dynamic  processes.  We  concluded  that  predicting  motion  from  static  diagrams  engages 
students’  mental  animation  processes,  including  spatial  visualization,  and  provides  them 
with  information  about  what  they  do  and  do  not  understand  about  how  the  machine 
works.  Verbal  instmction  provides  information  that  is  not  easily  conununicated  in 


graphics  and  directs  students’  attention  to  the  relevant  information  in  static  and  animated 
diagrams.  Our  research  suggests  that  an  understanding  of  students’  mental  animation 
abilities  is  ^  important  component  of  a  theory  of  learning  from  external  animations. 

In  these  experiments  we  examined  the  effects  of  spatial  ability  and  prior 
knowledge  of  machines  on  comprehension  of  the  various  instructional  media.  While  there 
were  main  effects  such  that  those  with  high-spatial  ability  and  more  knowledge  had 
superior  comprehension  outcomes,  there  was  no  evidence  for  aptitude-treatment 
interactions,  i.e.  that  different  formats  of  instruction  were  more  or  less  effective  for 
students  with  different  abilities  or  knowledge. 

6.3  Research  on  Comprehension  of  Hypermedia  Instructional  Manuals  * 

Finally,  in  several  experiments,  we  examined  individual  differences  in  learning 
from  hypermedia  instructional  manuals  that  included  several  different  sections  aimed  to 
address  different  phases  in  the  comprehension  of  mechanical  systems.  These  experiemtns 
were  based  on  hypermedia  presentations  developed  by  Narayanan  &  Hegarty  (1998; 

2002)  with  funding  from  the  parent  grant.  Details  of  tiie  structure  of  these  manuals  are 
described  by  Narayanan  &  Hegarty,  2002). 

We  conducted  several  experiments  comparing  these  hypermedia  manuals  to  (1) 
informationally  equivalent  printed  manuals  (2)  animations  typical  of  extant  research  and 
commercial  CD-ROMS,  and  (3)  mixed-mode  explanations  extracted  from  commercial 
books.  In  all  of  these  experiments  we  found  main  effects  of  spatial  ability  and  prior 
knowledge  in  that  students  with  more  ability  or  knowledge  tended  to  have  better 
comprehension,  but  there  were  no  interactions  of  individual  differences  with  format  of 
instruction.  The  most  striking  results  of  these  experiments  were  that  that  manuals 
designed  according  to  our  guidelines  were  more  effective  than  manuals  that  did  not 
conform  to  these  guidelines  -  both  commercial  and  research  products  (Narayanan  & 
Hegarty,  2002;  Hegarty,  Narayanan  &  Freitas,  2002). 

In  collaboration  with  researchers  at  the  Applied  Physics  Laboratory,  University  of 
Washington,  we  conducted  an  additional  experiment  that  investigated  whether  people 
learn  more  from  multimedia  presentations  explaining  how  machines  work  if  the 
presentations  show  realistic,  3d  diagrams  of  the  mechanical  systems  rather  than  2D  cross- 
sectional  views.  We  had  two  professionally  authored  multimedia  presentations  built  for 
testing.  These  manuals,  one  using  2D  and  the  other  using  3D  graphics,  explain  a 
relatively  complex  but  familiar  device:  the  flushing  cistern.  In  an  experimental  evaluation 
of  these  manuals  at  UCSB,  we  varied  (1)  whether  students  learned  from  the  hypermedia 
manuals  or  merely  from  a  labeled  diagram  of  the  machine  and  (2)  whether  they  received 
2-d  or  3-d  diagrams  in  their  training.  Results  indicated  that  those  who  received  the 
hypermedia  visualizations  had  superior  comprehension  of  the  machine,  but  there  was  no 
difference  between  those  who  received  2-d  and  3-d  views.  There  were  again  no 
interactions  of  ability  or  knowledge  with  format  of  instruction. 

Finally  in  an  experiment  on  interactivity  we  compared  the  effects  of  guided  versus 


free  navigation  of  our  hypermedia  manual  on  learning  outcomes  (Hegarty,  Narayanan  & 
Freitas,  2002).  Results  indicated  no  differences  in  comprehension  between  participants 
who  were  guided  to  view  die  sections  of  our  hypermedia  manual  in  an  order  prescribed 
by  our  design  guidelines  and  those  who  were  free  to  view  the  sections  in  any  order. 
Moreover,  fliere  were  few  differences  in  how  people  in  the  guided  and  free  navigation 
conditions  actually  interacted  with  the  materials.  Even  those  who  were  free  to  navigate 
the  manual  in  any  order  tended  to  examine  the  materials  in  a  linear  order.  In  these 
experiments  there  were  no  significant  effects  of  spatial  ability  on  interaction  with  the 
hypermedia  presentations. 

7.  Main  ResultsTBest  Accomplishments 

We  found  that  visual-spatial  representations  alone,  including  realistic,  interactive 
animations,  have  limited  utility  for  training  novices  about  how  machines  work  and  that  it 
is  important  to  augment  these  representations  with  verbal  instructions  that  draw  people’s 
attention  to  the  relevant  information  in  diagrams  and  describe  non-visible  entities  (such 
as  forces)  that  are  not  visible  in  realistic  visualizations  of  machines. 

In  several  experiments,  we  found  that  comprehension  of  mechanical  systems  from 
multimedia  and  hypermedia  systems  was  better  for  students  with  high  spatial  ability  than 
for  students  with  low  spatial  ability.  However  there  was  no  evidence  that  spatial  ability 
interacted  with  the  format  of  instruction,  that  is,  there  was  no  evidence  that  different 
formats  of  instruction  were  more  or  less  effective  for  students  with  different  abilities. 

In  all  of  our  experiments,  we  found  that  learning  from  static  diagrams  and  text 
was  equivalent  to  learning  from  animated  diagrams  and  commentaries,  when  these 
materials  contained  the  same  information.  There  was  no  evidence  that  different  formats 
of  instruction  were  more  or  less  effective  for  individuals  with  different  abilities  and 
knowledge. 

8.  Impact/Applications 

Consistent  with  much  previous  research  (e.g.  Mayer  1991;  Hegarty  &Just,  1993) 
our  research  suggests  that  visual-spatial  representations  need  to  be  augmented  with  verbal 
instruction,  especially  when  designing  for  novices.  A  new  contribution  of  our  research  is 
to  specify  the  specific  ways  in  which  verbal  materials  can  augment  comprehension. 

These  are  (1)  to  direct  students  attention  to  the  most  relevant  information  in  the  diagram 
(2)  to  guide  them  in  “mentally  animating”  static  diagrams  and  (3)  to  provide  information 
about  non-visible  entities  such  as  forces. 

Our  research  provides  no  support  for  converting  static  printed  materials  to 
interactive  computer  presentations,  including  animations.  We  found  no  advantages  of 
animated  over  static  diagrams  in  our  research  and  no  advantage  of  interactive  over  non¬ 
interactive  computer  presentations. 


Our  research  provides  no  evidence  for  aptitude-treatment  interactions,  i.e.  that 
different  forms  of  instruction  are  more  or  less  suitable  for  students  with  different  abilities 
or  background  knowledge.  Students  with  higher  spatial  ability  or  more  relevant 
background  knowledge  learn  more  from  multimedia  and  hypermedia  presentations  in 
general,  but  ability  does  not  interact  with  format  of  instruction. 

9.  Technology  Transfer 

Our  main  means  of  facilitating  technology  transfer  has  been  through  research 
presentations  at  forums  attended  by  scientists  and  engineers  including  those  from 
industry  and  military.  Through  various  presentations  we  have  disseminated  the  results  of 
our  research  at  cognitive  psychology  and  education  conferences. 

We  have  also  presented  our  research  at  workshops  at  the  Naval  Pacific 
Meteorology  and  Oceanography  Center,  San  Diego,  CA,  and  at  ONR  Workshops  on 
Attention,  Perception  and  Data  Visualization. 

This  is  a  basic  research  project  and  therefore  no  actual  products  have  been 
produced.  However,  our  research  has  generated  a  set  of  guidelines  for  designing 
effective  multimedia  presentations  which  have  been  communicated  through  our 
publications  and  presentations. 

11.  Statistics 

UCSB:  Graduate  Students/Postdoctoral  scholars  supported  at  least  25%  from  this  grant 
for  at  least  1  qiuuter: 

Graduate  Students:  Non-minority  women:  1 

Minority  woman:  2 
Non-minority  man:  1 
Minority  man:  1 

12.  Journal  Articles 

Hegarty,  M.,  Kriz,  S.  &  Cate,  C.  (2003).  The  Roles  of  Mental  Animations  and 
External  Animations  in  Understanding  Mechanical  Systems.  Cognition  & 
Instruction  Cognition  &  Instruction,  21,  325-360.. 

Hegarty,  M.,  Quillici,  J.,  Narayanan,  N.  H.,  Holmquist,  S.,  &  Moreno,  R.  (1999). 
Multimedia  Instruction:  Lessons  from  Evaluation  of  a  Theory-based  Design. 
Joumalof  Educational  Multimedia  and  Hypermedia,  5,  1 1 9-150. 

Narayanan,  N.  H.  &  Hegarty,  M.  (1998).  On  designing  comprehensible  hypermedia 
manuals.  IntemationalJoumal  of  Human-Computer  Studies,  48,  267-301. 

Narayanan,  N.  H.  &  Hegarty,  M.  (2002).  Multimedia  design  for  communication  of 


dynamic  information.  International  Journal  of  Human-Computer  Studies,  5  7, 
279-315. 

13.  Books  or  Chapters 

Hegafty,  M.  Narayanan,  N.  H.  &  Freitas,  P.  (2002).  Understanding  machines  from 
multimedia  and  hypermedia  presentations.  In  J.  Otero,  A.  C.  Graesser  &  J.  Leon 
(Eds.).  The  Psychology  of  Science  Text  Comprehension.  Lawrence  Erlbaum 
Associates. 

Kriz,  S.  (2002)  Understanding  simultaneity  and  causality  in  static  diagrams  versus 
animation.  In  M.  Hegarty,  B.  Meyer  &  N.  H.  Narayanan  (Eds.)  (2002). 
Diagrammatic  Representation  and  Inference.  Proceedings  of  Diagrams  2002. 
Berlin:  Springer. 

14.  Technical  Reports 

none 

15.  Presentations 

Hegarty,  M.,  Quiliei,  J.,  Narayanan,  N.  H.,  Hohnquist,  S.,  &  Moreno,  R.  (1998,  April) 
Designing  hypermedia  manuals  to  explain  how  machines  work:  Lessons  from 
evaluation  of  a  theory-based  design.  Hyped-Media  to  Hyper-Media:  A  Workshop  at 
CHI'98:  ACM  SIGCHI  Conference  on  Hmnan  Factors  in  Computing  Systems,  Los 
Angeles,  CA. 

Hegarty,  M.,  Narayanan,  N.H.,  Cate,  C  &  Hohnquist,  S.  (1999,  July).  Understanding 
Machines  from  Diagrams,  Text  and  Hypermedia  Presentations.  Presented  at  the 
Annual  Conference  of  the  Society  for  Applied  Memory  and  Cognition.  Boulder,  CO. 

Kriz,  S.  (2002,  April).  Understanding  simultaneity  and  causality  in  static  diagrams  versus 
animation.  Paper  presented  at  the  Diagrams  2002  conference,  Calaway  Gardens,  GA. 

Kriz,  S.  &  Hegarty,  M.  (2003,  July)  Staring  into  Space:  Evaluating  mechanical  motion, 
mental  animation  and  eye  movements.  Paper  presented  at  the  European  Eye 
Movement  Conference,  Aberdeen,  Scotland. 

16.  Patents  Issued  and  Pending 

None 

17.  Honors 

Mary  Hegarty  was  promoted  from  Associate  Professor  to  Professor,  effective  July  1, 

2000.  - 


MaryHegarty  .  Elected  Fellow  of  the  American  Psychological  Society,  2001. 

18.  Additional  References  (not  listed  above). 

Hegarty,  M.  (1992).  Mental  animation:  Inferring  motion  from  static  diagrams  of 
mechanical  systems.  Journal  of  Experimental  Psychology:  Learning,  Memory  and 
Cognition,  18,  \0M-\\02. 

Mayer,  R.  E.  (2001).  Multimedia  Learning.  New  York:  Cambridge  University  Press. 

Tversky,  B.,  Morrison,  J.  B.  &  Betrancourt,  M.  (2002).  Animation:  Can  it  facilitate? 
IntemationalJoumal  of  Human  Computer  Studies,  57,241-262. 

19.  Related  Projects 

Hegarty,  M.  Analysis  and  Synthesis  of  Hypermedia  Visualizations  Office  of  Naval 
Research  1997-2002,  N000149610525. 

Naryayanan,  N.  H.  Eye  Tracking  Equipment  for  Analysis  and  Synthesis  of  Human 
Interactions  with  Hypermedia  Information  Visualizations,  2000-2001,  $127,200,  Defense 
Universities  Research  Instrumentation  Program  (DUREP)  Grant  managed  by  ONR. 


